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Abstract. We propose a novel method for inferring refinement types of 
higher-order functional programs. The main advantage of the proposed 
method is that it can infer maximally preferred (i.e., Pareto optimal) 
refinement types with respect to a user-specified preference order. The 
flexible optimization of refinement types enabled by the proposed method 
paves the way for interesting applications, such as inferring most-general 
characterization of inputs for which a given program satisfies (or vi¬ 
olates) a given safety (or termination) property. Our method reduces 
such a type optimization problem to a Horn constraint optimization 
problem by using a new refinement type system that can flexibly rea¬ 
son about non-determinism in programs. Our method then solves the 
constraint optimization problem by repeatedly improving a current solu¬ 
tion until convergence via template-based invariant generation. We have 
implemented a prototype inference system based on our method, and 
obtained promising results in preliminary experiments. 


1 Introduction 

Refinement types mm have been applied to safety verification of higher-order 
functional programs. Some existing tools EimUM enable fully automated 
verification by refinement type inference based on invariant generation tech¬ 
niques such as abstract interpretation, predicate abstraction, and CEGAR. The 
goal of these tools is to infer refinement types precise enough to verify a given 
safety specification. Therefore, types inferred by these tools are often too specific 
to the particular specification, and hence have limited applications. 

To remedy the limitation, we propose a novel refinement type inference 
method that can infer maximally preferred (i.e., Pareto optimal) refinement 
types with respect to a user-specified preference order. For example, let us con¬ 
sider the following summation function (in OCaml syntax) 

let rec sum x = if x = 0 then 0 else x + sum (x - 1) 

A refinement type of sum is of the form (x : {x : int | P(x)}) —» {y : int | Q{x , y)} 
Here, P(x) and Q(x,y) respectively represent pre and post conditions of sum. 
Note that the postcondition Q(x , y) can refer to the argument x as well as the 
return value y. Suppose that we want to infer a maximally-weak predicate for 
P and maximally-strong predicate for Q within a given underlying theory. Our 


method allows us to specify such preferences as the following constraints for type 
optimization 

maximize(P), minimize(Q). 

Here, maximize(P) (resp. minimize(Q)) means that the set of the models of P(x) 
(resp. Q(x,y)) should be maximized (resp. minimized). Our method then infers 
a Pareto optimal refinement type with respect to the given preferences. 

In general, however, this kind of multi-objective optimization involves a 
trade-off among the optimization constraints. In the above example, P may 
not be weakened without also weakening Q. Hence, there often exist multiple 
optima. Actually, all the following are Pareto optimal refinement types of sum0 
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Our method further allows us to specify a priority order on the predicate 
variables P and Q. If P is given a higher priority over Q (we write P d Q), 
our method infers the type (2), whereas we obtain the type (3) if Q d P. 
Interestingly, (3) expresses that sum is non-terminating for any input x < 0. 

The flexible optimization of refinement types enabled by our method paves 
the way for interesting applications, such as inferring most-general characteri¬ 
zation of inputs for which a given program satisfies (or violates) a given safety 
(or termination) property. Furthermore, our method can infer an upper bound 
of the number of recursive calls if the program is terminating, and can find a 
minimal-length counterexample path if the program violates a safety property. 

Internally, our method reduces such a refinement type optimization prob¬ 
lem to a constraint optimization problem where the constraints are expressed 
as existentially quantified Horn clauses over predicate variables HHnmsi. The 
constraint generation here is based on a new refinement type system that can 
reason about (angelic and demonic) non-determinism in programs. Our method 
then solves the constraint optimization problem by repeatedly improving a cur¬ 
rent solution until convergence. The constraint optimization here is based on an 
extension of template-based invariant generation m to existentially quantified 
Horn clause constraints and prioritized multi-objective optimization. 

The rest of the paper is organized as follows. Sections [2] and [3] respectively 
formalize our target language and its refinement type system. The applications 
of refinement type optimization are explained in Section |4] Section [5] formalizes 
Horn constraint optimization problems and the reduction from type optimiza¬ 
tion problems. Section [6] proposes our Horn constraint optimization method. 
Section [3 reports on a prototype implementation of our method and the re¬ 
sults of preliminary experiments. We compare our method with related work in 
Section [5] and conclude the paper in Section [3 

1 Here, we use quantifier-free linear arithmetic as the underlying theory and consider 
only atomic predicates for P and Q. 
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Fig. 1. The operational semantics of our language L 


2 Target Language L 

This section introduces a higher-order call-by-value functional language L , which 
is the target of our refinement type optimization. The syntax is defined as follows. 

(programs) D ::= {/,; x t = ej”1 1 

(expressions) e x | e u | n | op(ui,..., u 0r ( 0 p)) I ifz u then ei else e 2 
| let x = ei in e 2 | let a: = in e let x = *3 in e 
(values) v x | x v \ n 

(eval. contexts) E ::= [ ] | E v \ let x = E in e 

Here, x and / are meta-variables ranging over variables, n and op respectively 
represent integer constants and operations such as + and >. ar(op) expresses 
the arity of op. We write x (resp. v) for a sequence of variables (resp. values). For 
simplicity of the presentation, the language L has integers as the only data type. 
We encode Boolean values true and false respectively as non-zero integers and 
0. A program is expressed as a set {/) Xi = 1 of function definitions. We 

define dom({/i x, = e,}™ x ) = {/ 1 ,..., / m }- We assume that a value x v satisfies 
1 < |u| < ar(f), where |u| represents the length of the sequence v. 

The call-by-value operational semantics of L is given in Figure [1] Here, Jop] 
represents the integer function denoted by op. Both expressions let x = *v in e 
and let x = *3 in e generate a random integer n, bind x to it, and evalu¬ 
ate e. They are, however, interpreted differently in our refinement type system 
(see Section [3]). These expressions are used to model external functions with¬ 
out definitions and non-deterministic behavior caused by external inputs (e.g., 
user inputs, interrupts, and so on). We write —>* D to denote the reflexive and 
transitive closure of — >d- 


3 Refinement Type System for L 

In this section, we introduce a refinement type system for L that can reason about 
non-determinism in programs. We then formalize refinement type optimization 
problems (in Section [3.1II . which generalize ordinary type inference problems. 




The syntax of our refinement type system is defined as follows. 

(refinement types) r ::= {z \ <fi} \ (z : ti) —> r 2 
(type environments) F ::= 0 | F, x : r | F, <fi 

(formulas) </> ::= t\ < t 2 | T | _L | -><f> | </>i A 4> 2 \ 4>i V </> 2 | </>i =► <h 

(terms) t ::= n \ x \ t± + <2 | ra • t 

(predicates) p ::= Aa;.^ 

An integer refinement type {z \ (f>} equipped with a formula <fi for type refinement 
represents the type of integers x that satisfy (f>. The scope of x is within (j>. 
We often abbreviate {x | T} as int. A function refinement type (z : t\) T 2 
represents the type of functions that take an argument x of the type t\ and 
return a value of the type T 2 . Here, T 2 may depend on the argument x and the 

scope of x is within 72 . For example, (z : int) —>• {y \ y > x} is the type of 

functions whose return value y is always greater than the argument x. We often 
write Jvs(t) to denote the set of free variables occurring in r. We define F(z) = r 
if x : t € r and dom(T') = {x \ x : t £ T}. 

In this paper, we adopt formulas <j) of the quantifier-free theory of linear 
integer arithmetic (QFLIA) for type refinement. We write \= (j> if a formula <j> is 
valid in QFLIA. Formulas T and _L respectively represent the tautology and the 
contradiction. Note that atomic formulas <1 < (resp. t\ = < 2 ) can be encoded 
as t\ + 1 < t 2 (resp. t\ < f 2 A t% <ti) in QFLIA. 

The inference rules of our refinement type system are shown in Figure [2j 
Here, a type judgment b D : T means that a program D is well-typed under a 
refinement type environment r. A type judgment r h e : r indicates that an 
expression e has a refinement type t under r. A subtype judgment f h q <: T 2 
states that t\ is a subtype of r 2 under r. [T] occurring in the rules ISub and 
Rand 3 is defined by [0] = T, [T, x : {u \ <(>}J = [T] A [x/v\(j>, JF, x : (v : 
ri) —y T 2 ] = [FJ, and [F, = JF] A c/>. In the rule Op, [[op] Ty represents a 

refinement type of op that soundly abstracts the behavior of the function [op]. 
For example, [+] Ty = (z : int) —» (y : int) -A- {z \ z = x + y}. All the rules 
except RandV and Rand 3 for random integer generation are essentially the 
same as the previous ones [18]. The rule RandV requires e to have r for any 
randomly generated integer x. Therefore, e is type-checked against r under a 
type environment that assigns int to x. By contrast, the rule Rand 3 requires 
e to have r for some randomly generated integer x. Hence, e is type-checked 
against r under a type environment that assigns a type {x \ <fi} to x for some 
cj) such that fvs((j>) C dom(F) U {z} and \= [F] =>■ 3x.<f>. For example, z : 
int h let y = *3 in z + y : {r \ r = 0 } is derivable because we can derive 
z : int, y : {y \ y = —z} b z + y : {r \ r = 0}. Thus, our new type system 
allows us to reason about both angelic *3 and demonic *v non-determinism in 
higher-order functional programs. 

We now discuss properties of our new refinement type system. We can prove 
the following progress theorem in a standard manner. 

Theorem 1 (Progress). Suppose that we have b D : F, dom(F) = dom(D), 
and r be: r. Then, either e is a value or e —>d r! for some e'. 
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Fig. 2. The inference rules of our refinement type system 


We can also show the substitution lemma and the type preservation theorem in 
a similar manner to |18j . 

Lemma 1 (Substitution). If T b v : t' and r,x : t',F' b e : r, then 
r, [v/x\r' b [v/x\e : [v/x\t. 

Theorem 2 (Preservation). Suppose that we have b D : T and r b e : r. If 
e is of the form let x = *3 in eo, then we get F b e' : r for some e' suc/i that 
e —e/. Otherwise, we get F b e 7 : r /or any e' such that e — e'. 

Proof. We prove the theorem by induction on the derivation of T b e : r. We only 
show the case for the rule Rand 3 below. The other cases are similar to [18] . By 
Rand 3 , we have e = let x = *3 in eo, fvs{(j >) C dom(r') U {x}, |= [T] =>■ 3x.<p, 
r,x : {x | </>} b e 0 : t, and 2 ^ fvs(r). It then follows from |= [TJ => 3 x.cj) that 
there is an integer n such that |= [1] A 1 : = n / By the rule E-Rand 3 , we 
get e — >d [n/x\eQ = e'. By the rules Int and Sub, we obtain r b n : {x \ (j)}. 
Thus, we get T b e! : r by Lemma [T] F, x : {a; | </>} b eo : t, and x ^ fvs(r). □ 

3.1 Refinement Type Optimization Problems 

We now define refinement type optimization problems, which generalize refine¬ 
ment type inference problems addressed by previous work [9i n0l[l5HT9] . 















We first introduce the notion of refinement type templates. A refinement type 
template of a function / is the refinement type obtained from the ordinary ML- 
style type of / by replacing each base type int with an integer refinement type 
{v | P(x, v)} for some fresh predicate variable P that represents an unknown 
predicate to be inferred, and each function type Tf —>• T 2 with a (dependent) 
function refinement type (x : t 4 ) —> r 2 . For example, from an ML-style type 
(int —» int) — > int —> int, we obtain the following template. 

(/ : (x\ : {xi | .Pi(xi)}) -»■ {x 2 | P 2 (xi,x 2 )}) -> 

(x 3 : {x 3 | P 3 (x 3 )}) -> {x 4 | P 4 (x 3 ,x 4 )} 

Note here that the first argument / is not passed as an argument to P 3 and 
P 4 because / is of a function type and never occurs in QFLIA formulas for 
type refinement. A refinement type template of a program D with dom(P) = 
{/i,...,/ m } is the refinement type environment Pd = /1 : : r m , 

where each r, is the refinement type template of /,;. We write jws(Pd) for the 
set of predicate variables that occur in Pd- A predicate substitution 6 for Pd 
is a map from each P € pvs(rn) to a closed predicate Ax 4 ,..., x or (p).(/>, where 
ar(P) represents the arity of P. We write 0 Pd to denote the application of a 
substitution 9 to Pd- We also write dom(0) to represent the domain of 9. 

We can define ordinary refinement type inference problems as follows. 

Definition 1 (Refinement Type Inference). A refinement type inference 
problem of a program D is a problem to find a predicate substitution 9 such that 
b D : 9r D . 

We now generalize refinement type inference problems to optimization problems. 


Definition 2 (Refinement Type Optimization). Let D be a program, -< be 
a strict partial order on predicate substitutions, and 0 = {9 | b D : 0Pd}- A 
predicate substitution 9 £ 0 is called Pareto optimal with respect to -< if there 
is no 9' € 0 such that 9' -< 9. A refinement type optimization problem (D, -<) 
is a problem to find a Pareto optimal substitution 9 £ 0 with respect to -<. 

In the remainder of the paper, we will often consider type optimization problems 
extended with user-specified constraints and/or templates for some predicate 
variables (see SectionS] for examples and Section [5] for formal definitions). 

The above definition of type optimization problems is abstract in the sense 
that is only required to be a strict partial order on predicate substitutions. We 
below introduce an example concrete order, which is already explained informally 
in Section[l]and adopted in our prototype implementation described in Section!?] 
The order is defined by two kinds of optimization constraints: the optimization 
direction (i.e. minimize/maximize) and the priority order on predicate variables. 

Definition 3. Suppose that 

— V = {Pi,... ,P m } is a subset of pvs(Pd), 


— p is a map from each predicate variable in V to an optimization direction d 
that is either f (for maximization) or f (for minimization), and 

— n is a strict total order on V that expresses the priority^ We below assume 
that Pi C • ■ • C P m - 

We define a strict partial order ^( p ,c) on predicate substitutions that respects p 
and C as the following lexicographic order: 

01 ~<(p,C) 02 e {1, .. . , TO} . 0\ (Pi) -<p(Pi) 02 (Pi')hidj < i- 01 (Pj) =p(Pj) 02 (Pj) 

Here, a strict partial order ~<d and an equivalence relation =d on predicates are 
defined as follows. 

~ Pi <d P2 Pi P2 A p 2 7<d Pi, 

~ Pi =d P2 <S==> Pi <d P2 A p 2 <d Pi, 

— Xx.(j )2 -<=>-|= 4>2 => </> i, and Xx.cfi ^ 4 . Ax .02 4=4-|= 4 >2- 

Example 1. Recall the function sum and its type template with the predicate 
variables P, Q in Section [T| Let us consider optimization constraints p(P) =t> 
p(Q) =1, and PcQ, and predicate substitutions 

— 0i = {P Xx. x = 0, Q !->• Ax, y. y = x}, 

— 02 = {P4 Ax. T, Q i->- Ax, y.y > 0}, and 

— 03 = {P i->- Ax. x < 0, Q Ax, y. _L}. 

We then have 62 -<(p,c) ^1 and 02 ^(p.c) 03 , because (Xx. T) ~<f (Ax. x = 0) and 
(Ax. T) (Ax. x < 0). □ 

4 Applications of Refinement Type Optimization 

In this section, we present applications of refinement type optimization to the 
problems of proving safety (in Section [4. II) and termination (in Section [O]), and 
disproving safety (in Section 14.41) and termination (in Section 14.21) of programs 
in the language L. In particular, we discuss precondition inference, namely, in¬ 
ference of most-general characterization of inputs for which a given program 
satisfies (or violates) a given safety (or termination) property. 

4.1 Proving Safety 

We explain how to formalize, as a type optimization problem, a problem of 
inferring maximally-weak precondition under which a given program satisfies 
a given postcondition. For example, let us consider the following terminating 
version of sum. 

2 If C were partial, the relation defined shortly would not be a strict partial 

order. Our implementation described in Section [7] uses topological sort to obtain a 
strict total order O from a user-specified partial one. 



let rec sum 1 x = if x <= 0 then 0 else x + sum’ (x- 1 ) 

In our framework, a problem to infer a maximally-weak precondition on the argu¬ 
ment x for a postcondition x = sum' x is expressed as a type optimization prob¬ 
lem to infer sun/’s refinement type of the form (x : {x | P(x)}) —>■ {y \ x = y} 
under an optimization constraint maximize(P). Our type optimization method 
described in Sections 15.21 and [G] infers the following type. 

(x : {x | 0 < x < 1}) {y \ x = y} 

This type says that the postcondition holds if the actual argument x is 0 or 1. 

Example 2 (Hihger-Order Function). For an example of a higher-order function, 
consider the following. 

let rec repeat f n e = if n<=0 then e else repeat f (n-1) (f e) 

By inferring repeat’s refinement type of the form 

{f :(x:{x | Pi(x)}) ->■ {y \ P 2 (x,y)}) -)• (n : int) -)• (e : {e | P 3 (n,e)}) -t {r | r 

under optimization constraints p(-Pi) =1, p{P 2 ) = p(P'i) =ti and P 3 C P 2 Cl Pi, 
our type optimization method obtains 

(/ : (x : {x | x > 0}) —> {y \ y > 0}) —> (n : int) —> (e : {e | e > 0}) —» {r \ r > 0} 

Thus, type optimization can be applied to infer maximally-weak refinement types 
of (possibly higher-order) arguments that are sufficient for the function to satisfy 
a given postcondition. □ 

4.2 Disproving Termination 

In a similar manner to Section 14.11 we can apply type optimization to the prob¬ 
lems of inferring maximally-weak precondition for a given program to violate 
the termination property. For example, consider the function sum in Section [TJ 
For disproving termination of sum, we infer sum’s refinement type of the form 
(x : {x | P(x)}) —>• {y | _L} under an optimization constraint maximize(P). Our 
type optimization method infers the following type. 

(2 : {x | x < 0}) {y | 1} 

The type expresses the fact that no value is returned by sum (i.e., sum is non¬ 
terminating) if the actual argument x satisfies x < 0 . 

Example 3 (Non-Deterministic Function). For an example of non-deterministic 
function, let us consider a problem of disproving termination of the following. 

let rec f x = let n = read_int () in if n<0 then x else f x 

Here, read_int () is a function to get an integer value from the user and is 
modeled as *3 in our language L. Note that the termination of f does not 
depend on the argument x but user inputs n. Actually, our type optimization 
method successfully disproves termination of f by inferring a refinement type 
(x : int) —» {y \ _L} for f and {n \ n > 0} for the user inputs n. This means that 
f is non-terminating if the user always inputs some non-negative integer. □ 


4.3 Proving Termination 


Refinement type optimization can also be applied to bounds analysis for inferring 
upper bounds of the number of recursive calls. Our bounds analysis for functional 
programs is inspired by a program transformation approach to bounds analysis 
for imperative programs urn- Let us consider sum in Section [0 By inserting 
additional parameters i and c to the definition of sum, we obtain 

let rec sum_t x i c = if x=0 then 0 else x + sum_t (x-1) i (c+1) 

Here, i and c respectively represent the initial value of the argument x and the 
number of recursive calls so far. For proving termination of sum, we infer sum_t’s 
refinement type of the form 


(x : {x | P(x}) —> (i : int) —»• (c : {c | Inv(x, i, c)}) —i int 


under optimization constraints maximize(P ), minimize(Bnd ), P IZ Bnd, and 
additional constraints on the predicate variables P , Bnd , Inv 


Vx, i, c. ( Inv(x , i, c) •$= c = 0 A i = x) 

Vx, i, c. ( Bnd(i , c) •<= P(x) A Inv(x, i, c)) 


(4) 

(5) 


Here, Bnd(i,c) is intended to represent the bounds of the number c of recursive 
calls of sum with respect to the initial value i of the argument x. We therefore 
assume that Bnd(i , c) is of the form 0 < c < kg + k\ ■ i, where Icq, k\ represent 
unknown coefficients to be inferred. The constraint (4) is necessary to express the 
meaning of the inserted parameters i and c. The constraint (5) is also necessary 
to ensure that the bounds Bnd(i,c) is implied by a precondition P(x) and an 
invariant Inv(x,i,c) of sum. Our type optimization method then infers 

(x : {x | x > 0}) —> ( i : int) —>-(c:{c|x<iAi = x + c}) —i int 

and Bnd(i,c) = 0 < c < i. Thus, we can conclude that sum is terminating for 
any input x > 0 because the number c of recursive calls is bounded from above 
by the initial value i of the argument x. 

Interestingly, we can infer a precondition for minimizing the number of re¬ 
cursive calls of sum by replacing the priority constraint P C Bnd with Bnd C P 
and adding an additional constraint 3x.P(x) (to avoid a meaningless solution 
P(x) = _L). In fact, our type optimization method obtains 


(x : {x | x = 0}) — > ( i : int) — > (c : {c | c = 0}) —» int 


and Bnd(i, c) = c = 0. Therefore, we can conclude that the minimum number of 
recursive calls is 0 when the actual argument x is 0. 

We expect that our bounds analysis for functional programs can further be 
extended to infer non-linear upper bounds by adopting ideas from an elaborate 
transformation for bounds analysis of imperative programs [7i . 


4.4 Disproving Safety 


We can use the same technique in Section 14.31 to infer maximally-weak precon¬ 
dition for a given program to violate a given postcondition. For example, let us 
consider again the function sum. A problem to infer a maximally-weak precondi¬ 
tion on the argument x for violating a postcondition sum x > 2 can be reduced 
to a problem to infer sum_t’s refinement type of the form 

(x : {x | P(x)}) -)• ( i : int) ->• (c : {c | Inv(x, i, c)}) -)• {y \ ~^(y > 2)} 

under the same constraints for bounds analysis in Section 14.31 The refinement 
type optimization method then obtains 

(x : {x | 0 < x < 1}) — > ( i : int) — >(c:{c\0<xAi = x + c}) —> {y \ ~>(y > 2)} 

and Bnd(i, c) = 0 < c < i. This result says that if the actual argument x is 0 or 
1, then sum terminates and returns some integer y that violates y > 2. In other 
words, x = 0,1 are counterexamples to the postcondition sum x > 2. 

We can instead find a minimal-length counterexample pa l i Q to the post¬ 
condition sum x > 2 by just replacing the priority constraint P c Bnd with 
Bnd □ P and adding an additional constraint 3 x.P[x). Our type optimization 
method then infers 

(x : {x | x = 0}) —> ( i : int) —>(c:{c|0<xAi = x + c}) —> {y \ ~^{y > 2)} 

and Bnd(i,c) = c = 0. From the result, we can conclude that a minimal-length 
counterexample path is obtained when the actual argument x is 0. 

5 Horn Constraint Optimization and Reduction from 
Refinement Type Optimization 

We reduce refinement type optimization problems into constraint optimization 
problems subject to existentially-quantified Horn clauses mum- We first for¬ 
malize Horn constraint optimization problems (in Section 15.11) and then explain 
the reduction (in Section [5721) . 

5.1 Horn Constraint Optimization Problems 

Existentially-Quantified Horn Clause Constraint Sets (3HCCSs) over QFLIA are 
defined as follows. 


(3HCCSs) H ::= {hci ,..., hc m } 

(Horn clauses) he ::= h <= b 

(heads) h ::= P(t) \ (f) \ 3 x.(P(t) A 
(bodies) b ::= Pi(ti) A ... A P m (t m ) A 4> 

Here, minimality is with respect to the number of recursive calls within the path. 
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We write pvs(H) for the set of predicate variables that occur in H. 

A predicate substitution 6 for an 3HCCS T~L is a map from each P G pvs(7~L) 
to a closed predicate Axi,... ,x ar (py(f>. We write 0-p for the set of predicate 
substitutions for H. We call a substitution 0 is a solution of H if for each he GW., 
|= dhc. For a subset 0 C 0 U , we call a substitution 6 G 0 is a 0-restricted 
solution if 0 is a solution of H. Our constraint optimization method described 
in Section [G] is designed to find a 0-restricted solution for some 0 consisting of 
substitutions that map each predicate variable to a predicate with a bounded 
number of conjunctions and disjunctions. In particular, we often use 

0atom = |p Axx,... ,X or (p).n 0 + i7?r£ P) ni • Xi > 0 I P G pvs(H )| 
consisting of atomic predicate substitutions. 

Example 4- Recall the function sum and the predicate substitutions 63 , 82,83 in 
Example [T] Our method reduces a type optimization problem for sum into a 
constraint optimization problem for the following HCCS 7t SU m (the explanation 
of the reduction is deferred to Section [Oil . 

( Q{ x, 0) -t= P(x) As = 0, P(x — 1) <= P{x) Ai/0, 

\ Q(x, x + y) 4= P(x) A Q(x - 1, y) A x ^ 0 

Here, 83 is a solution of P sum , and 0 2 and 63 are @ atom -restricted solutions of 
"Hsum- If we fix Q{x,y) = _L (i.e., infer sum’s type of the form (x : {x \ P(x)}) —> 
{y | _L}) for disproving termination of sum as in Section 14.21 we obtain the fol¬ 
lowing HCCS 

{± •<= P(x) A x = 0, P{x — 1) -4= P(x) Ai/0} 

has, for example, 0 atom -restricted solutions {P Ax.x < 0} and {P 
Ax.x < —100}. □ 

We now define Horn constraint optimization problems for 3HCCSs. 

Definition 4. Let W be an 3HCCS and -< be a strict partial order on predicate 
substitutions. A solution 6 of W is called Pareto optimal with respect to -< if 
there is no solution O' of W such that O' -< 0. A Horn constraint optimization 
problem ('H, -<) is a problem to find a Pareto optimal solution 0 with respect to 

A 0-restricted Horn constraint optimization problem is a Horn constraint 
optimization problem with the notion of solutions replaced by 0 -restricted solu¬ 
tions. 

Example 5. Recall 7t S um a-nd its solutions 0 \, 02,03 in Example[TJ Let us consider a 
Horn constraint optimization problem (7t sum , ^( p ,c)) where p{P) = t- p(Q) = 
and Q C P. We have O 3 -<( p ,c) 0\ and 83 ^( p ,c) 82 - In fact, 83 is a Pareto 
optimal solution of 7t S um with respect to ^( p ,c)- □ 


In general, an 3HCCS P may not have a Pareto optimal solution with re¬ 
spect to even though P has a solution. For example, consider a Horn 

constraint optimization problem (P svm , -<( P|C )) where p{P) =t, p(Q) =4-, and 
P C Q. Because the semantically optimal solution Q(x, y) = y = x ( x + 1 '> i s not 
expressible in QFLIA, it must be approximated, for example, as Q(x , y) = y > 
0 A y > x A y > 2a;—1. The approximated solution, however, is not Pareto 
optimal because we can always get a better approximation like Q(x,y) = y > 
0Ay>xAy>2x — 1 A y > 3x — 3 if we use more conjunctions. 

We can, however, show that an 3HCCS has a <9 a t om -restricted Pareto optimal 
solution with respect to -<(p,c) if it has a © a t om -restricted solution. For the above 
example, 9 2 in Example Q] is a @ a t om -restricted Pareto optimal solution. 

Lemma 2. Suppose that an 3HCCS P has a & atom,-restricted solution and for 
any P such that p(P) =4-, P is not existentially quantified in P. It then follows 
that P has a 0atom-restricted Pareto optimal solution with respect to -<( Pj p. 

Proof Sketch. We prove the lemma by contradiction. Suppose that P has a 
(9 atom -restricted solution but no Pareto optimal one. It then follows that there 
exist an infinite descending chain 9 1 9 2 P( p ,xz) • ■ • °f <9atom-restricted 

solutions and a predicate variable P such that 

- V* > 1. 8 t (P) ^ p(P) 8 i+ 1 (P) A VQ c P. 8 i(Q) = P(Q) 8 i+ 1 (Q) and 

— no 0atom-restricted solution is a lower bound of the chain. 

The key observations here are that the half-spaces represented by &i(P), 82 (P), ■ ■ ■ 
are parallel, and for some k > 0 that depends on 2 ar ( p ' 1 and the largest absolute 
value of coefficients in 8 \{P), the distance di between &i(P) and 9 i+ k(P ) are di > 
1 for all i > 1 , because of the strictness of y p (p) and the discreteness of integers. 
By continuity of P , P has a 0 a tom-restricted solution 9 such that 9(P) = Xx.T 
if p(P) =f and 8 (P) = Ax.T if p(P) =f, and VQ C P. 8 {Q) = P (Q) 9i(Q). 9 is 
obviously a lower bound of the chain. Thus, a contradiction is obtained. □ 

5.2 Reduction from Refinement Type Optimization 

Our method reduces a refinement type optimization problem into an Horn con¬ 
straint optimization problem in a similar manner to the previous refinement 
type inference method [TS]. Given a program D 1 our method first prepares a 
refinement type template Pd of I? as well as, for each expression of the form 
let x — *3 in e, a refinement type template {x | P(y, x)} of x, where P is a fresh 
predicate variable and y is the sequence of all integer variables in the scope. Our 
method then generates an 3HCCS by type-checking D against ro and collecting 
the proof obligations of the forms [f] A => <4 > 2 and [FJ =>■ 3v.cj) respectively 
from each application of the rules ISub and Rand 3. We write Gen(D, Tp) to 
denote the 3HCCS thus generated from D and F p . 

We can show the soundness of our reduction in the same way as in [TS]. 

Theorem 3 (Soundness of Reduction). Let ( D , -<) be a refinement type op¬ 
timization problem and Pd be a refinement type template of D. If 9 is a Pareto 
optimal solution of Gen(D , Fu), then 9 is a solution of ( D , -<). 



1: procedure Optimize^, -<) 

2: match Solve("H) with 

3: Unknown — > return Unknown 


4: | NoSol —> return NoSol 

5: | Sol(9 0 ) ->• 

6 : 0 := 6 > O ; 

7: while true do 


9 

10 

11 

12 

13 


8: let T-L' = Improve^ (9, H) in 

9: match Solve( 77 / ) with 


Unknown —> return Sol(9) 

| NoSol —> return OptSol(9) 


\ Sol(6') -> 6> := 9 1 


end 


Fig. 3. Pseudo-code of the constraint optimization method for BHCCSs 

6 Horn Constraint Optimization Method 

In this section, we describe our Horn constraint optimization method for 3HCCSs. 
The method repeatedly improves a current solution until convergence. The pseudo¬ 
code of the method is shown in Figure [3] The procedure Optimize for Horn con¬ 
straint optimization takes a (©-restricted) 3HCCS optimization problem ( ki , -<) 
and returns any of the following: Unknown (which means the existence of a solu¬ 
tion is unknown), NoSol (which means no solution exists), Sol(9 ) (which means 
9 is a possibly non-Pareto optimal solution), or OptSol{9 ) (which means 9 is a 
Pareto optimal solution). The sub-procedure Solve for Horn constraint solving 
takes an 3HCCS T~L and returns any of Unknown , NoSol, or Sol (9). The detailed 
description of Solve is deferred to Section RTT1 

Optimize first calls Solve to find an initial solution 9q of T-L (line [2]). Opti¬ 
mize returns Unknown if Solve returns Unknown (line [3]) and NoSol if Solve 
returns NoSol (line 2]). Otherwise (line [5]), Optimize repeatedly improves a cur¬ 
rent solution 9 starting from 9o until convergence (lines [HI fl3l) . To improve 9, 
we call a sub-procedure IMPROVE^ {9, T-L) for generating an 3HCCS T-L' from H 
by adding constraints that require any solution 9' of TV satisfies 9' <9 (line [8]). 
Optimize then calls Solve to find a solution of T-L'. If Solve returns Unknown, 
Optimize returns Sol(9) as a (possibly non-Pareto optimal) solution (line [Toll . If 
Solve returns NoSol, it is the case that no improvement is possible, and hence 
the current solution 9 is Pareto optimal. Thus, Optimize returns OptSol{9 ) (line 
mD. Otherwise, we obtain an improved solution 9' -< 9, and Optimize updates 
the current solution 9 with 9' and repeats the improvement process (line [12]). 

Example 6 . Recall 'H^ am in Example [I] and consider an optimization problem 
CH^m, -<(c,p)) where p(P) =f. We below explain how Optimize^^,, 

) proceeds. First, Optimize calls the sub-procedure Solve to find an initial 
solution of Hsim (e.g., = {P ^ Acc.-L}). Optimize then calls the sub-procedure 
Improve^ (9 0 , H^m) an( i obtains an 3HCCS H 1 = / H^ m D{P(x) <= _L, 3a;.P(:r)A 
-i-L}. Note that %' requires that for any solution 9 of , 9{P) -< p (p) 9 q{P) = 



Ax._L. Optimize then calls Solve(?T) to find an improved solution of H (e.g., 

9 1 = {P i— y Ax.x < 0}). In the next iteration, Optimize returns 9 \ as a Pareto 
optimal solution because IMPROVE^ (9\ , has no solution. □ 

We now discuss properties of the procedure Optimize under the assumption 
of the correctness of the sub-procedure Solve (i.e., 9 is a ©-restricted solution of 
H. if Solve (H) returns Sol{9 ), and TL has no ©-restricted solution if Solve (H) 
returns NoSol). The following theorem states the correctness of Optimize. 

Theorem 4 (Correctness of the Procedure Optimize). Let (H,^) be a 
O-restricted Horn constraint optimization problem. If Optimize (H, -<) returns 
OptSol(9) (resp. Sol(9)), 9 is a Pareto optimal (resp. possibly non-Pareto opti¬ 
mal) O-restricted solution of H with respect to -<. 

The following theorem states the termination of Optimize for © otom -restricted 
Horn constraint optimization problems. 

Theorem 5 (Termination of the Procedure Optimize). Let (H, -<(tz,p)) 
be a 0 a t 0 m-restricted Horn constraint optimization problem. It then follows that 
Optimize( 7t,^( CiP )) always terminates if the sub-procedure Solve preferen¬ 
tially returns solutions having smaller absolute values of coefficients. 

Proof. Recall the proof sketch of Lemma [2] Any infinite descending chain of 
©atom-restricted solutions for a predicate variable P with respect to -<(c,p) has 
a limit Ax.T if p(P) =t and A2?._L if p(P) =f. Because Aa?._L (resp. Alr.T) is 
expressed as an atomic predicate Xx. — 1 > 0 (resp. Alr.O > 0) having absolute 
values of coefficients not greater than 1 and the number of such predicates is 
finite, the limit is guaranteed to be reached in a finite number of iterations. □ 


6.1 Sub-Procedure Solve for Solving 3HCCSs 

The pseudo-code of the sub-procedure Solve for solving 3HCCSs is presented 
in Figure |4] Here, Solve uses existing template-based invariant generation tech¬ 
niques based on Farkas’ lemma Hi and 3HCCS solving techniques based on 
Skolemization mmm- Solve first generates a template substitution 9 that 
maps each predicate variable in pvs(fH) to a template atomic predicate with un¬ 
known coefficients Co,..., c Qr .(p) (line [2]) 0 Solve then applies 9 to H and obtains 
a verification condition of the form 3c.Vx3y.(f> without predicate variables (line 
0. Solve applies Skolemization minis] to the condition and obtains a sim¬ 
plified condition of the form 3c, zMx.cj)' (line0. Solve further applies Farkas’ 
lemma [3j[8] to eliminate the universal quantifiers and obtains a condition of the 
form 3 c,z,ui.(f>" (line [5]). Solve then uses an off-the-shelf SMT solver to find a 
satisfying assignment to 4>" (line [6]). If such an assignment a is found, Solve 

4 In this way, the particular code is specialized to solve 0 a i om -restricted Horn con¬ 
straint optimization problems. To solve ©-restricted optimization problems for other 
©, we need here to generate templates that conform to the shape of substitutions in 
© instead. Our implementation in Section [3 iteratively increases the template size. 



1: procedure Solve(P) 

2: let 6 = |p h->- Aai.co + £°f=^Ci ■ Xi > 0 | P £ pvs(H ) j in 

3: let Jc.Vx3y.(j> = 3c./\ hceH Vfvs(hc). 8 (hc) in 

4: let 3c,'z.yx.cf>' = apply Skolemization to 3c.yx.3y.cf) in 

5: let 3c,'z,w.cf)" = apply Farkas’ lemma to 3c,'zNx.cf>' in 

6: match SMT(</) ,/ ) with 

7: Unknown —» return Unknown 

8: | Satfa) — » return Sol(a( 8 )) 

9: | Unsat —»■ match SMT(V5;.3y.0) with 

10: Unknown —» return Unknown 

11: | Sat(a) —» return Sol(a(8)) 

12: | Unsat —> return NoSol 

Fig. 4. Pseudo-code of the constraint solving method for BHCCSs based on template- 
based invariant generation 


returns a (9) as a solution (line [S]) . Solve returns Unknown if the SMT solver 
returns Unknown (line [7]). Otherwise (no assignment is found) If] Solve uses the 
SMT solver again to find a satisfying assignment a to Vx.3y.cj) (line [5]). If such a 
a is found, Solve returns <r(9) as a solution (line HIT) . Solve returns Unknown 
if Unknown is returned (line HOD and NoSol if Unsat is returned (line 1121) . 

Example 7. We explain how Solve proceeds for %' in Example [6] Solve first 
generates a template substitution 9 = {P Ax.Cq + C\ ■ x > 0} with unknown 
coefficients Cq, C\ and applies 9 to . As a result, we get a verification condition 


3c 0 , c\. 


Vx. 


(1 4= cq + ci • x > 0 A x = 0) A 


(co + ci • (x — 1) > 0 4= Co + ci • x > 0 A x ^ 0) 
3x. cq + ci • x > 0 


By applying Farkas’ lemma, we obtain 
'3w\ ,W 2 ,W 3 > 0. 

3rU4, W5,Wq > 0. 


3co, c\. 


( 3w\ W2,W3 > 0 . (co • Wl < — 1 A Cl • W\ + W2 — W3 = 0 ) A 

( — 1 - CO + Cl) • W4 + Co • W5 — W6 < -1 a' 
ci • (—W4 + wj 5 ) + we = 0 
( — 1 — Co + Cl) • W7 + Co • Ws — Wg < — 1 a' 
Cl • (~w 7 + w s ) -w 9=0 


3w 7, Ws,Wg > 0. 

y 3x. Co + ci • x > 0 
By using an SMT solver, we obtain, for example, a satisfying assignment 


_ f Co I-A — 1, Cl —1, Wl 1-4 1, W 2 1, W3 t-4 0, 1 

( W4 1—^ 0 , W3 1— 1 , wq 1—^ 1 , W7 1 —v 1 , w% 1 —y 0 , wg 1 —y 1 J 

Thus, Solve returns a(9) = {Pn Ax. — 1 — x > 0} = 9\ in Example [6] □ 

5 Note here that even though no assignment is found, H may have a 0 a i om -restricted 
solution because Farkas’ lemma is not complete for QFLIA formulas 130 and 
Skolemization of 3c.yx.3y.cf> into 3c, 'z.Vx.cf)' here assumes that y are expressed as 
linear expressions over x pirnis]. 






Table 1 . The results of a non-termination verification benchmark set used in mm- 
The results for CppInv, T2-TACAS, and TNT are according to Larraz et al. 13 . The 
result for MoCHi is according to HU- 



Verified 

TimeOut 

Other 

Our tool 

41 

27 

13 

CppInv [13j; 

70 

6 

5 

T2-TACAS [2] 

51 

0 

30 

TNT [5j 

19 

3 

59 

MoCHi [U] 

48 

26 
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The following theorem states the correctness of the sub-procedure Solve. 

Lemma 3 (Correctness of the Sub-Procedure Solve). LeVH be an 3HCCS. 

9 is a O a tom-restricted solution ofH if Solve (fH) returns Sol(9), and H has no 
0atom,-restricted solution if Solve (fH) returns NoSol. 

Note that the sub-procedure Solve described above does not necessarily 
satisfy the assumption of Theorem [5] We can, however, extend Solve to satisfy 
the assumption by bounding the absolute values of unknown coefficients and 
iteratively incrementing the bounds in SMT solving. 


7 Implementation and Experiments 

We have implemented a prototype refinement type optimization tool based on 
our method. Our tool takes OCaml programs and uses Z3 0] as the underlying 
SMT solver in the sub-procedure Solve. We conducted preliminary experiments 
for each application presented in Section [U All the experiments were conducted 
on a machine with Intel Core i7-4650U 1.70GHz, 8GB of RAM. 

The experimental results are summarized in Tables Q] and 0 Table [T] shows 
the results of an existing first-order non-termination verification benchmark set 
used in mmm- Because the original benchmark set was written in the input 
language of T2 (http://mmjb.gith.ub. io/T2/), we used an OCaml translation 
of the benchmark set provided by in. Our tool was able to successfully disprove 
termination of 41 programs (out of 81) in the time limit of 100 seconds. Our 
prototype tool was not the best but performed well compared to the state-of- 
the-art tools dedicated to non-termination verification. 

Table [2] shows the results of maximally-weak precondition inference for prov¬ 
ing safety and termination, and disproving safety and termination. In the col¬ 
umn ^Iterations, > 2 represents that the 3-rd iteration timed out and possibly 
non-Pareto optimal solution was inferred by our tool. We used non-termination 
(resp. termination) verification benchmarks for higher-order programs from [TTj 
(resp. [T2]). The results show that our method is also effective for safety and non¬ 
termination verification of higher-order programs. Our prototype tool, however, 
could be optimized further to speed up termination and non-safety verification. 














Table 2. The results of maximally-weak precondition inference for proving/disproving 
safety/termination. 


Program 

Applications 

/^Iterations 

Time (ms) 

f ixpoint_nonterm [11] 

Disprove Termination 

1 

2,020 

fib_CPS_nonterm [TTi 

Disprove Termination 

1 

5,023 

indirect_e ill 

Disprove Termination 

1 

1,083 

indirectHO_e 1111 

Disprove Termination 

1 

2,434 

loopHQ |TT] 

Disprove Termination 

1 

1,642 

foldr nonterm |11| 

Disprove Termination 

1 

4,904 

repeat (Sec. 14.11) 

Prove Safety 

4 

948 

sum_geq3 

Prove Safety 

4 

2,654 

append 

Prove Safety 

5 

20,352 

append : 12J 

Prove Termination 

6 

26,786 

zip [12] 

Prove Termination 

13 

76,641 

sum' (Sec. 14.11) 

Prove Safety 

3 

1,856 

sum (Sec. 14.21) 

Disprove Termination 

1 

174 

sum_t (Sec. 14.31) 

Prove Termination(P c Inv) 

4 

56,042 

sum_t (Sec. 14.31) 

Prove Termination(/nn C P) 

3 

6,628 

sum_t (Sec. 14.41) 

Disprove Safety(P C Inv) 

> 2 

18,009 

sum_t (Sec. 14.41) 

Disprove Safety(/nn c P) 

> 2 

16,540 


8 Related Work 

Type inference problems of refinement type systems mm have been intensively 
studied mmsasi- To our knowledge, this paper is the first to address type 
optimization problems, which generalize ordinary type inference problems. As 
we saw in Sections [4] and [3 this generalization enables significantly wider appli¬ 
cations in the verification of higher-order functional programs. 

For imperative programs, Gulwani et al. have proposed a template-based 
method to infer maximally-weak pre and maximally-strong post conditions [8]. 
Their method, however, cannot directly handle higher-order functional pro¬ 
grams, (angelic and demonic) non-determinism in programs, and prioritized 
multi-objective optimization, which are all handled by our new method. 

Internally, our method reduces a type optimization problem to a constraint 
optimization problem subject to an existentially quantified Horn clause con¬ 
straint set (3HCCS). Constraint solving problems for EIHCCSs have been studied 
by recent work Bunns]. They, however, do not address constraint optimization 
problems. The goal of our constraint optimization is to maximize/minimize the 
set of the models for each predicate variable occurring in the given 3HCCS. Thus, 
our constraint optimization problems are different from Max-SMT [l4j problems 
whose goal is to minimize the sum of the penalty of unsatisfied clauses. 

9 Conclusion 

We have generalized refinement type inference problems to type optimization 
problems, and presented interesting applications enabled by type optimization 












to inferring most-general characterization of inputs for which a given functional 
program satisfies (or violates) a given safety (or termination) property. We have 
also proposed a refinement type optimization method based on template-based 
invariant generation. We have implemented our method and confirmed by ex¬ 
periments that the proposed method is promising for the applications. 
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